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Appendix A. About the study 

This appendix provides background on the current reading proficiency among students in the Commonwealth of 
the Northern Mariana Islands (CNMI) and the factors that might be associated with it and discusses Head Start 
and other CNMI early care and education programs. 


Reading proficiency among students on the Commonwealth of the Northern Mariana Islands 


In 2019, 24 percent of grade 3 students in the CNMI Public School System demonstrated proficiency on 
standardized measures of reading skills in English (Commonwealth of the Northern Mariana Islands Office of 
Accountability, Research, and Evaluation, 2019). Research indicates that meeting reading benchmarks in grade 3 
positively correlates with later academic outcomes (Hein et al., 2013) and postsecondary enrollment. One study 
found that students who met grade 3 reading proficiency benchmarks graduated from high school and enrolled 
in college at higher rates than students who did not meet the benchmarks (Lesnick et al., 2010). The CNMI Public 
School System aims to support students not only in meeting academic achievement standards but also in acquiring 
“essential skills for lifelong learning, and ha[ving] the ability to transition into higher education and/or the 
workforce” (Commonwealth of the Northern Mariana Islands Public School System, 2021b). This means that 
gaining further insight into the relationship between reading performance and characteristics of grade 3 CNMI 
students is of critical interest. 


Many CNMI students are members of historically under-resourced groups. For example, 44 percent of families in 
the CNMI have a household income below the federal poverty level compared with 12 percent of all U.S. families 
(Commonwealth of the Northern Mariana Islands Department of Commerce, 2018). Prior research indicates that 
students who live in lower income households or who reside in under-resourced communities are at higher risk 
for difficulties in school, including lower reading and math achievement (Burchinal et al., 2011; Pungello et al., 
1996; Welsh et al., 2010). One study that used data from the U.S. Department of Education’s Early Childhood 
Longitudinal Study Kindergarten Cohort found that by the time children from the most disadvantaged 
socioeconomic group began kindergarten, they had cognitive scores that were 60 percent lower on average than 
those of children in the least disadvantaged socioeconomic group (Lee & Burkam, 2002). 
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Other student demographic characteristics have been shown to influence student performance. For example, a 
U.S. Government Accountability Office study conducted in the continental United States found that school 
mobility was associated with lower performance on measures of academic skills and a higher probability of 
dropping out of school (Ashby, 2010). The same study showed that by grade 8, 70 percent of students had 
attended two or fewer schools, 18 percent had attended three schools, and 13 percent had attended four schools 
(Ashby, 2010). A similar study conducted on early grades found that 36 percent of students changed schools once 
between kindergarten and grade 3, whereas 55 percent of students did not change schools at all (Burkam et al., 
2009). Although the current study did not explore the reasons why CNMI students changed schools, prior research 
indicates that mobility is often associated with economic or social challenges (Mehana & Reynolds, 2004). 
Enrolling in a new school might compound these issues by creating gaps in attendance, introducing inconsistencies 
in exposure to curricula and pedagogy, and disrupting relationships with teachers and peers (Paik & Philips, 2002). 


CNMI stakeholders have suggested that enrollment in an early childhood education program might be associated 
with grade 3 reading proficiency. Growing evidence suggests a positive relationship between enrollment in a high- 
quality early childhood education program and reading skills development (Yoshikawa et al., 2016). Although 
some evidence indicates that the impacts of programs like Head Start do not persist beyond kindergarten, other 
studies suggest that the impacts might correlate with later academic outcomes (Deming, 2009). For example, 
middle school students who had enrolled in Head Start were more likely than students who had not to have higher 
math scores on state tests and were less likely to be retained by the time they reached grade 8 (Philips et al., 
2016). 


The extent to which results from the continental United States hold true in the CNMI is unclear. CNMI Public 
School System stakeholders are interested in exploring how student demographic characteristics (such as 
mobility) and other education experiences (such as Head Start enrollment) are related to grade 3 reading 
proficiency. A better understanding of these relationships is expected to provide CNMI Public School System 
stakeholders with additional information to inform current and future initiatives, instruction, and student 
supports. 


Head Start and other early care and education programs 


Families in the CNMI have access to early childcare services through the CNMI Public School System, 25 licensed 
facilities, and nine license-exempt home care providers (Commonwealth of the Northern Mariana Islands 
Preschool Development Grant Birth through Five, 2020). The CNMI Head Start program has been in place for more 
than 30 years. As of 2019, nine Head Start centers were in operation across the islands of Saipan, Tinian, and Rota, 
with morning, afternoon, and full-day sessions available, depending on the number of students enrolled 
(Commonwealth of the Northern Mariana Islands Office of Accountability, Research, and Evaluation, 2019). The 
extent to which CNMI children not enrolled in Head Start attend other preschool programs is unknown because 
programs are spread out across the three islands and data collection is inconsistent and not centralized 
(Commonwealth of the Northern Mariana Islands Preschool Development Grant Birth through Five, 2020). 


Head Start is the largest and longest running publicly funded early childhood education program for low-income 
families with preschool-age children in the United States. Founded in 1965 as part of President Lyndon Johnson’s 
Great Society program, Head Start provides more than 900,000 children and their families access to preschool 
education and a range of health, nutrition, and education services each year. More than 1,600 local grantees 
administer Head Start programs across all 50 states, the District of Columbia, and U.S. territories. 


Many state and local governments supplement federal Head Start programs and offer non—Head Start preschool 
programs to a substantial number of children. According to the National Institute for Early Education Research, 
total state funding for preschool in 2015 was $6.2 billion (Barnett & Friedman-Krauss, 2016). These funds provide 
support for nearly 1.4 million children to attend state-funded preschool programs, with approximately 13 percent 
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of three-year-old children and 38 percent of four-year-old children enrolled in either Head Start or other publicly 
funded preschool programs (Barnett & Friedman-Krauss, 2016). 


Despite the increased availability of publicly funded preschool programs, not all eligible children participate in 
them. In 2015 Head Start provided services to 18 percent of low-income three-year-old children and 21 percent 
of low-income four-year-old children (Barnett & Friedman-Krauss, 2016). Children from low-income families are 
less likely to attend preschool than their peers from higher income families (Crosnoe et al., 2016; Lopez et al., 
2017). Lower enrollment rates among children from low-income families are worth addressing, as growing 
evidence suggests a relationship between participating in a high-quality early childhood education program and 
starting kindergarten with the skills and abilities that support school success (Yoshikawa et al., 2016). Although 
the CNMI Head Start program has worked to recruit and enroll children—and many children in the CNMI are 
eligible for enrollment—enrollment rates have fallen in recent years (Commonwealth of the Northern Mariana 
Islands Office of Accountability, Research, and Evaluation, 2019). Even though Head Start eligibility requirements 
limit the number of participating families based on income levels (Sec. 645. [42 U.S.C. 9840]), a small percentage 
of CNMI families enroll in the program regardless of income (Commonwealth of the Northern Mariana Islands 
Public School System, 2021a; Early Childhood Learning and Knowledge Center, 2021). Per Head Start policy, this 
number cannot exceed 10 percent of enrolled children (Early Childhood Learning and Knowledge Center, 2021). 


CNMI stakeholders are interested in examining the relationship between enrollment in the Head Start program 
and grade 3 reading performance, among other potential predictors. Because Head Start is under the auspices of 
CNMI Public School System, stakeholders are particularly interested in understanding outcomes associated with 
Head Start, but there is limited research on the program regarding its possible relationship to CNMI student K-12 
outcomes. 
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Appendix B. Methods 


This appendix describes the study’s data sources, the process for matching and cleaning data, measures and the 
process for recoding variable, and analytic methods. 


Data sources 


District staff from the Commonwealth of the Northern Mariana Islands (CNMI) Public School System, including the 
Head Start program, provided the study team with the administrative data files used to conduct the analyses in 
this study. At the time of the study requests for CNMI data, furloughs at the CNMI Public School System and the 
spread of COVID-19 limited staff capacity to compile and share school- and district-level data with the study team. 
The data files lacked consistent information on teacher education, teacher core academic skills, student 
absenteeism, student English learner status, and student eligibility for special education services. As a result, the 
study was unable to analyze these data. 


Student-level administrative data were collected by schools and shared with CNMI Public School System staff, who 
then shared the data files with the study team. The information was provided for all students enrolled in 
kindergarten, grade 1, grade 2, and grade 3 during the 2011/12—2018/19 school years. The study team received 
information on student enrollment in Head Start for the 2009/10-2015/16 school years. In addition to the 
administrative data, CNMI public schools collected student performance data on the ACT Aspire reading 
assessment. School leaders shared the files from 2011/12—2018/19 with the CNMI Public School System staff, who 
then provided the information to the study team.? 


Matching and cleaning data 

The student identifier in the student-level files was not consistent across years. To link student records across 
years, the study team used students’ full name and date of birth and the distance function (COMPGED) in SAS. 
This function generates a distance measure that characterizes the difference between two strings. A distance of 
0 represents an exact match, and any distance more than 5 was reviewed by the study team before a match was 
confirmed. This approach enabled 98 percent of student records to be matched. 


Measures and variable recoding 

The measures used in the study are discussed below. The description of each measure includes the methods used 
to recode the variables prior to analysis so that the coefficient for a given categorical variable could be compared 
with the grand mean outcome rather than with the outcomes for another categorical variable. Deviation coding 
was used for categorical variables. The specific values for the predictor variables used in the research question 
analyses are presented in table B1. 


Outcome variable 


Grade 3 reading proficiency. The reading proficiency of CNMI students was measured at the end of grade 3 using 
the reading domain of the ACT Aspire assessment. ACT Aspire is a computer-based standardized benchmark 
assessment that measures academic skills in five domains (English, math, reading, science, and writing) in grades 
3-10. ACT Aspire reports an overall scale score and readiness and scale scores in four domains (writing is collapsed 
within the other content-area domains). The study team used the readiness and scaled scores from the reading 
domain of the spring 2015-19 grade 3 administrations as the outcome measure. The reading domain covers 


1 The CNMI Public School System also shared teacher-level data, such as teacher evaluation scores, but the teacher-level data were not 
used due to high rates of missing data and other quality concerns. 
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informational and literacy text skills relating to identifying key ideas and details in text, analyzing text structure 
and purpose, and integrating knowledge and ideas from text. Data were provided as a categorical variable 
indicating whether a student scored significantly below (Needs Support), below (Close), at (Ready), or above 
(Exceeding) grade-level benchmarks for proficiency. The study team then converted that categorical variable to a 
binary variable indicating whether a student demonstrated proficiency (received a benchmark score of Ready or 
Exceeding) or did not demonstrate proficiency (received a benchmark score of Close or Needs Support) in reading 
in grade 3. 


Student demographic characteristics 


Gender. CNMI Public School System administrative records reported two categories of gender—female and male. 
The analyses used the record for students’ kindergarten year. This variable was deviation coded as —1 (male) and 
1 (female). 


Free or reduced-price lunch receipt. Student status indicating receipt of meals provided during the school day for 
free or at a reduced cost. Students are eligible for free or reduced-price lunch if their household income is at or 
below 185 percent of the federal income poverty level and their families enroll them in the program. Once 
enrolled, individual student data records are marked as receiving the benefit. There might be families who were 
eligible for the benefit but did not apply for it. Only children of families who applied received the benefit. CNMI 
Public School System administrative records report whether a student received free or reduced-price lunch until 
the 2018/19 school year, when the CNMI Public School System started providing free meals for all students.* The 
analyses used the record for students’ kindergarten year. This variable was deviation coded as —1 (student did not 
receive free or reduced-price lunch) and 1 (student received free or reduced-price lunch). 


Age at kindergarten entry. CNMI Public School System administrative records included student date of birth. The 
study team calculated age in both months and days as of August 5 of the year a student started kindergarten. This 
variable was mean centered. Five students in the sample were very young at kindergarten entry (between four 
years, two months and four years, five months). The study team conducted a sensitivity analysis that removed 
extreme age values, and the results did not change substantially (see appendix D). 


Ethnicity. Data on student ethnicity were derived from CNMI Public School System administrative files. CNMI 
Public School System data files included records for 18 ethnicity categories, which families report on school entry 
forms. The study team collapsed these into three categories: Chamorro or Carolinian;? Filipino; and other, which 
combined the remaining ethnicity categories (Bangladeshi, Chinese, Chuukese, Japanese, Korean, Kosraean, 
Marshallese, Other, Other Asian, Other Pacific Islander, Palauan, Pohnpeian, White, and Yapese). This variable 
was deviation coded into three separate ethnicity values. Category A was coded as —1 (Other), 0 (Chamorro or 
Carolinian), and 1 (Filipino). Category B was coded as —1 (Other), 0 (Filipino), and 1 (Chamorro or Carolinian). 


Education experiences 


Head Start enrollment. Information on enrollment in Head Start was collected by the CNMI Public School System 
when students enrolled in kindergarten. The study team used the information in these administrative records to 
create a binary variable equal to 1 if a student had enrolled in Head Start for one or more years and —1 if a student 
had not enrolled in Head Start. 


Mobility. For each student the study team identified the school of record from CNMI Public School System 
administrative records for each year between kindergarten and grade 3. The study team created a dichotomous 
variable equal to 0 if the student changed schools one or more times during the period and 1 if the student was 


? The study did not include kindergarten records from 2018/19. 
3 This report uses Chamorro and Carolinian to align with the terms used by the CNMI Public School System. Some community members in 
CNMI, however, may prefer CHamoru and Refaluwasch respectively. 
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enrolled in the same school for the entire period. This variable was then deviation coded as —1 (student attended 
two or more schools between kindergarten and grade 3) and 1 (student attended only one school between 
kindergarten and grade 3). 


Table B1. Predictor variable coding in analytic models 


Measure Centering method Value label WENIVT=) 
Gender Deviation coding Male -1 
Female 1 
Free or reduced-price lunch Deviation coding Student did not receive free or reduced-price lunch in -1 
receipt in kindergarten kindergarten 
Student received free or reduced-price lunch in kindergarten 1 
Age at kindergarten entry Mean centered Continuous age centered on sample mean —14.7 to 8.1 
(months) 
Ethnicity category A Deviation coding Other =il 
Chamorro or Carolinian 0 
Filipino 1 
Ethnicity category B Deviation coding Other -1 
Filipino 0 
Chamorro or Carolinian 1 
Head Start enrollment Deviation coding Student had not enrolled in Head Start -1 
Student had enrolled in Head Start for one or more years 1 
School mobility Deviation coding Student attended two or more schools between kindergarten -1 
and grade 3 
Student attended only one school between kindergarten and 1 
grade 3 


Source: Authors’ compilation. 


Analytic methods 
This section describes the data analyses for the study. 


Hierarchical modeling. The study team used hierarchical logistic regression modeling (HLM) to estimate the 
associations between student demographic characteristics and education experiences and the outcome of 
interest. Because students were nested in five cohorts and schools, it was necessary to account for the nesting 
structure of the dataset using HLM (Hedeker & Gibbons, 2006; Raudenbush & Bryk, 2002). 


The two-level HLM model for the study’s research question is as follows.* For a dichotomous outcome variable Yij 
~ Bernoulli distribution (77;;), the logit link function is: 


nj = B1 + Bo (Agei;) + B3 (FRPLi;) + B,( Gender; ; ) + Bs (Ethnicity; ; ) + Bo (Education; ;) + uj + ej 
where nj; = In (4), 7;; denotes the probability that student / in cohort j received a benchmark score of Ready 
aij 


or Exceeding on the grade 3 reading assessment; Age;;,FRPL;;,Gender;;, and Ethnicity;; are child-level 


ip 
demographic characteristics variables; Education;; is a vector of education experience variables, including 


mobility and Head Start enrollment; 62 to 6, are regression coefficients for each corresponding demographic 


4 A three-level HLM model that included school as a random intercept was tried first, but it was removed because the model would not 
converge and there was no estimate for school. 
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characteristic variable; u; represents the random effects for cohort j; and ej represents the random effects for 
student ij. 


The study team conducted the analyses in R version 4.04 (R Core Team, 2021) and used the Ime4 package version 
1.1-26 (Bates et al., 2015) to conduct the logistic HLM analyses, the mice package version 3.13.0 (van Buuren & 
Groothuis-Oudshoorn, 2011) to impute and pool the HLM results, and the emmeans package version 1.5.5-1 
(Lenth, 2021) to calculate predicted probabilities from the HLM models. 


Predicted probabilities. The study team estimated the least square means of the group effect on the logit scale 
using the emmeans package in R (Lenth, 2021). This produced an estimated probability by mapping the logits onto 
the probability scale. Marginal probabilities were than predicted for each subgroup that was found to be 
statistically significant for a given outcome. For continuous predictor variables the probabilities were calculated 
at each quartile of the variable to show how the probabilities changed across the range of observed values. Other 
variables in the model were held at their mean. 


Missing data. Data were missing for 17 percent of the analytic sample (table B2). 


Table B2. Missing data for key variables in research question analyses 


Number of students 
with a grade 3 IN TUT a] okey ato) mers 215 Percentage of cases 


ACT Aspire with data with data 
Variable assessment record on this variable on this variable 


Outcome: Grade 3 reading proficiency 2,095 2,095 100.0 


Gender 2,095 2,094 99.9 
Free or reduced-price lunch receipt in kindergarten 2,095 2,095 100.0 
Age at kindergarten entry 2,095 2,072 98.8 
Ethnicity 2,095 2,093 99.9 


Head Start enrollment 2,095 2,095 100.0 
School mobility 2,095 1,760 84.0 
All variables used in the analyses 2,095 1,742 83.2 


Source: Authors’ analysis based on 2011-19 administrative data files provided by the Commonwealth of the Northern Mariana Islands Public School System. 


The sample of students used for the complete case analysis, in which students were dropped if they were missing 
data on any of the variables, was not statistically significantly different from the full sample with respect to any 
covariate. However, complete case analysis could have substantial weaknesses. First, listwise deletion limits the 
statistical power of the tests conducted because it uses only cases with complete information on all variables and 
thus limits the overall size of the sample (Allison, 2001; Olinsky et al., 2003). Second, if there is a systematic 
difference between the complete cases and the incomplete cases, the statistical inference from complete case 
analysis might not apply to the population of all cases. To eliminate this bias, the study team used multiple 
imputation for missing data as the main analysis. Missing data for predictors (student demographic characteristics 
and education experiences) were imputed using the predictive mean matching for continuous variables and 
polytomous regression imputation for unordered categorical variables via the R package mice (van Buuren & 
Groothuis-Oudshoorn, 2011). Data were not imputed for missing outcome measures. 


Multiple imputation inference involved three distinct phases: 


e The missing data were filled in 16 times based on the largest percentage of missingness for any of the 
variables, to generate 16 complete datasets. 
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e The 16 complete datasets were analyzed separately using HLM procedures. 


e The results from the complete datasets were combined for inference using the mice package’s pool() function, 
which combines the estimates and standard errors from analyses performed on each of the imputed datasets. 


Appendix C presents results from the imputed data as the main analysis in the report, and appendix D presents 
unimputed data for supplementary sensitivity analyses. 


Supplementary Head Start analyses 

Information on student enrollment in Head Start was collected by the CNMI Public School System when students 
enrolled in kindergarten. Fewer than 3 percent of cases were missing information for this analysis. The study team 
used the information in these administrative records to create a binary variable, equal to 1 if a student had 
enrolled in Head Start for at least one year and O if a student had not. Using a three-level logistic HLM model 
(student—cohort-school), the study team used student demographic characteristics (gender, free or reduced-price 
lunch receipt, age at kindergarten entry, and ethnicity) to predict Head Start enrollment. Further discussion of 
findings from these analyses can be found in appendix D. 
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Appendix C. Supporting analyses 


This appendix describes the study population’s sample descriptive statistics, as well as the results of the data 
analyses. 


Sample descriptives 
This section provides descriptive statistics of the study’s population. 


Table C1. Student demographic characteristics and education experiences, by grade 3 reading proficiency 
status (percent) 


1D) (e Matej mero) a = 
Scored Ready or Ready or Exceeding 
Exceeding on the on the ACT Aspire 


ACT Aspire Reading Reading 
assessment in assessment in 
All students grade 3 grade 3 
NVE-Val-]o)(=) (n = 2,095) (ee si0)) (et ol o)>)) 


All students 100 20.5 79.5 


Male 51.0 44.9 52.6 
Female 49.0 55.1 47.4 
Gender missing 0.1 0.0 0.1 
Did not receive free or reduced-price lunch at kindergarten entry 89.0 90.7 88.5 
Received free or reduced-price lunch at kindergarten entry 11.0 9.3 11.5 
Mean age at kindergarten entry? (months) 64.5 (3.7) 64.9 (3.7) 64.4 (3.7) 
Filipino 33.9 46.7 30.6 
Chamorro or Carolinian 47.2 35.8 50.1 
Other 18.8 17.2 19.2 
Ethnicity missing 0.1 0.2 0.1 
Had enrolled in Head Start 51.9 54.7 51.2 
Had not enrolled in Head Start 48.1 45.4 48.8 
Attended more than one school between kindergarten and grade 3 17.2 12.1 18.6 
Attended one school between kindergarten and grade 3 66.8 70.0 66.0 
Mobility missing 16.0 17.9 15.5 


Note: Percentages might not sum to 100 due to rounding. 

a. Information on age at kindergarten entry was missing for 32 students. About 15.6 percent of students who were missing information on age earned a 
score of Ready or Exceeding on the grade 3 reading assessment compared with 20.6 percent of students for whom information on age was not missing. 
Values in parentheses are standard deviations, which applied only to the continuous variable. 

Source: Authors’ analysis based on 2011-19 administrative data files provided by the Commonwealth of the Northern Mariana Islands Public School System. 


Results of the data analyses 


The section provides detailed results of the regression analyses examining the relationship between students’ 
demographic characteristics and education experiences and their grade 3 reading proficiency. 


Table C2. Hierarchical logistic regression model results for earning a score of Ready or Exceeding on the grade 
3 ACT Aspire reading assessment from analyses using multiple imputation of missing data 


Estimate 
NVF-Val-le)(=) (Koy -eyelo 9] Standard error 


Intercept —1.67*** 0.17 <.001 
Female 0.18** 0.06 <.01 
Received free or reduced-price lunch —0.21* 0.10 <.05 
Age at kindergarten entry 0.04* 0.02 <.05 
Filipino 0.44*** 0.08 <.001 
Chamorro or Carolinian —0.34*** 0.08 <.001 
Had enrolled in Head Start —0.03 0.06 61 
Attended one school between kindergarten and grade 3 0.18* 0.08 <.05 


Cohort 0.27 na na 


* Significant at p < .05; ** significant at p < .01; *** significant at p < .001. 

na is not applicable 

Note: n = 2,095. 

Source: Authors’ analysis based on Commonwealth of the Northern Mariana Islands Public School System 2011-19 administrative data files. 


Table C3. Predicted probability for earning a score of Ready or Exceeding on the grade 3 ACT Aspire reading 
assessment from analyses using multiple imputation of missing data 


Predicted 95 percent 
NYU] of:1 gel 0] o) probability confidence interval 
Gender—male 0.14 0.10-0.18 
Gender—female 0.18 0.14-0.24 
Received free or reduced-price lunch 0.13 0.09-0.19 
Did not receive free or reduced-price lunch 0.19 0.15-0.24 
Age at kindergarten entry—50 months 0.10 0.06-0.16 
Age at kindergarten entry—61 months 0.14 0.11-0.19 
Age at kindergarten entry—64 months 0.16 0.12-0.21 
Age at kindergarten entry—68 months 0.18 0.13-0.23 
Age at kindergarten entry—72 months 0.20 0.14-0.27 
Filipino 0.23 0.17-0.30 
Chamorro or Carolinian 0.12 0.09-0.16 
Other ethnicities 0.14 0.10-0.20 
Attended one school between kindergarten and grade 3 0.18 0.14-0.24 
Attended two or more schools between kindergarten and grade 3 0.14 0.09-0.19 


Note: n = 2,095. Only variables identified as statistically significant in table C2 are included in the table. The table includes predicted probabilities only for 
age values that are represented in figure 3 in the main report. 
Source: Authors’ analysis based on 2011-19 administrative data files provided by the Commonwealth of the Northern Mariana Islands Public School System. 
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Appendix D. Supplementary analyses 


This appendix provides the results of supplementary and sensitivity analyses. These analyses include the results 
of regression models using only complete case data, regression analyses examining the effects of extreme ages at 
kindergarten entry on the results where the values less than 2.5 and greater than 97.5 percentiles were removed, 
and regression moderator analyses. 


Because this study compared the school performance of students who had enrolled in Head Start and students 
who had not, it is important to first understand which children in the study sample had enrolled in Head Start. 
This appendix also provides supplementary analyses on the demographic characteristics of students who had 
enrolled in Head Start and students who had not, analyses that were done to understand this population of 
students and to inform the study’s main findings. 


Sensitivity analyses 

The results of the sensitivity analyses were all consistent with those of the analyses discussed in the main report. 
The results of the complete case analysis and the analysis using imputed data and the results of the analysis with 
all values for age at kindergarten entry and the analysis in which extreme age values were removed were similar. 
The differences in regression coefficients (also called regression estimates) on all predictors were less than or 
equal to 0.25 log odds. All analyses continued to indicate statistically significant higher probabilities of 
demonstrating reading proficiency in grade 3 for Filipino students, students who were older at kindergarten entry, 
female students, and students who did not receive free or reduced-priced lunch than for the average student 
(tables D1—D6). 


The study team also examined whether the relationship between Head Start enrollment and grade 3 reading 
proficiency differed for students from different ethnicity groups and by age at kindergarten entry. An interaction 
term between Head Start enrollment and ethnicity was added to the analytic model that used multiple imputation 
for missing data (table D5), and an interaction term between age at kindergarten entry and Head Start enrollment 
was added to the analytic model that included the interaction term between Head Start enrollment and ethnicity 
(table D6). The interaction terms were not statistically significant, indicating that neither ethnicity nor age at 
kindergarten entry moderated the relationship between Head Start enrollment and grade 3 reading proficiency. 
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Table D1. Hierarchical logistic regression model results for earning a score of Ready or Exceeding on the grade 
3 ACT Aspire reading assessment using multiple imputation for missing data and for the complete case 
analysis 
Multiple imputation Complete case 
(n = 2,095) (ee) 


Silalerlae| Simlarerlae| 


Variable Estimate error p-value Estimate error p-value 


Intercept —1.67*** 0.17 <.001 —1.73*** 0.19 <.001 
Female 0.18** 0.06 <.01 0.13* 0.06 <.05 
Received free or reduced-price lunch —0.21* 0.10 <.05 —0.25* 0.10 <.05 

Age at kindergarten entry (months) 0.04* 0.02 <.05 0.04* 0.02 <.05 
Filipino 0.44*** 0.08 <.001 0.49*** 0.12 <.001 
Chamorro or Carolinian —0.34*** 0.08 <.001 —0.31*** 0.13 <.001 
Had enrolled in Head Start —0.03 0.06 61 0.05 0.06 42 
Attended one school between kindergarten and grade 3 0.18* 0.08 <.05 0.17* 0.16 <.05 

JRandomette 

Cohort 0.27 na na 0.31 na na 


* Significant at p < .05; ** significant at p < .01; *** significant at p < .001. 
na is not applicable. 
Source: Authors’ analysis based on 2011-19 administrative data files provided by the Commonwealth of the Northern Mariana Islands Public School System. 


Table D2. Predicted probability of earning a score of Ready or Exceeding on the grade 3 ACT Aspire reading 
assessment using multiple imputation for missing data and for the complete case analysis (based on the 
hierarchical logistic regression model) 


Multiple imputation Complete case 
(n = 2,095) (a 9) 


95 percent 95 percent 


Predicted confidence Predicted confidence 
Subgroup probability Talkyavel probability interval 


Gender—male 0.14 0.10-0.18 0.13 0.10-0.19 
Gender—female 0.18 0.14-0.24 0.17 0.12-0.23 
Received free or reduced-price lunch 0.13 0.09-0.19 0.12 0.08-0.19 
Did not receive free or reduced-price lunch 0.19 0.15-0.24 0.18 0.14-0.24 
Age at kindergarten entry—50 months 0.10 0.06-0.16 0.09 0.05-0.16 
Age at kindergarten entry—61 months 0.14 0.11-0.19 0.14 0.10-0.19 
Age at kindergarten entry—64 months 0.16 0.12-0.21 0.15 0.11-0.20 
Age at kindergarten entry—68 months 0.18 0.13-0.23 0.17 0.12-0.23 
Age at kindergarten entry—72 months 0.20 0.14-0.27 0.19 0.13-0.27 
Filipino 0.23 0.17-0.30 0.23 0.16-0.30 
Chamorro or Carolinian 0.12 0.09-0.16 0.11 0.08-0.16 
Other ethnicities 0.14 0.10-0.20 0.13 0.09-0.19 
Attended one school between kindergarten and grade 3 0.18 0.14-0.24 0.17 0.13-0.23 
Attended two or more schools between kindergarten and grade 3 0.14 0.09-0.19 0.13 0.09-0.19 


Source: Authors’ analysis based on 2011-19 administrative data files provided by the Commonwealth of the Northern Mariana Islands Public School System. 


Table D3. Hierarchical logistic regression model results for earning a score of Ready or Exceeding on the grade 
3 ACT Aspire reading assessment for the full sample and excluding students with values for age at 
kindergarten entry in the <2.5 and >97.5 percentiles 


Excluding students with values for 
All values for age at kindergarten entry in the 
age at kindergarten entry included <2.5 and >97.5 percentiles 
(n = 2,659) (n = 2,594) 


S\ilalel-lac) Sii-laler-lae| 
NV FTar-le)(= Estimate error p-value Estimate error p-value 


Intercept —1.67*** 0.17 <.001 —1.65*** 0.17 <.001 
Female 0.18** 0.06 <.01 0.18** 0.06 <.01 
Received free or reduced-price lunch —0.21* 0.10 <.05 —0.20* 0.10 <.05 
Age at kindergarten entry (months) 0.04* 0.02 <.05 0.04* 0.02 <.05 
Filipino 0.44*** 0.08 <.001 0.45*** 0.08 <.001 
Chamorro or Carolinian —0.34*** 0.08 <.001 —0.34*** 0.08 <.001 
Had enrolled in Head Start —0.03 0.06 61 —0.04 0.06 52 
Attended one school between kindergarten and grade 3 0.18* 0.08 <.05 0.15* 0.08 .06 
SRandomeffe 

Cohort 0.27 na na 0.28 na na 


* Significant at p < .05; ** significant at p < .01; *** significant at p < .001. 
na is not applicable. 
Source: Authors’ analysis based on 2011-19 administrative data files provided by the Commonwealth of the Northern Mariana Islands Public School System. 


Table D4. Predicted probability of earning a score of Ready or Exceeding on the grade 3 ACT Aspire reading 
assessment for the full sample and excluding students with values for age at kindergarten entry in the <2.5 
and >97.5 percentiles 


All values for Excluding students with values 
age at kindergarten entry for age at kindergarten entry 
arellUreK=te} in the <2.5 and >97.5 percentiles 
(n = 2,659) (n = 2,594) 


95 percent 95 percent 
acrelranetel confidence actellaneye| confidence 
Subgroup probability interval probability Taeeave]| 


Gender—male 0.14 0.10—0.18 0.14 0.10-0.14 
Gender—female 0.18 0.14-0.24 0.19 0.14-0.25 
Received free or reduced-price lunch 0.13 0.09-0.19 0.14 0.09-0.20 
Did not receive free or reduced-price lunch 0.19 0.15-0.24 0.19 0.15-0.24 
Age at kindergarten entry—50 months 0.10 0.06-0.16 0.10 0.06—-0.16 
Age at kindergarten entry—61 months 0.14 0.11-0.19 0.14 0.11-0.19 
Age at kindergarten entry—64 months 0.16 0.12-0.21 0.16 0.12-0.21 
Age at kindergarten entry—68 months 0.18 0.13-0.23 0.18 0.13-0.24 
Age at kindergarten entry—72 months 0.20 0.14-0.27 0.20 0.15-0.28 
Filipino 0.23 0.17-0.30 0.23 0.17-0.30 
Chamorro or Carolinian 0.12 0.09-0.16 0.12 0.09-0.16 
Other ethnicities 0.14 0.10—-0.20 0.15 0.10-0.21 
Attended one school between kindergarten and grade 3 0.18 0.14-0.24 0.19 0.14-0.24 
Attended two or more schools between kindergarten and grade 3 0.14 0.09-0.19 0.14 0.09-0.19 


Source: Authors’ analysis based on 2011-19 administrative data files provided by the Commonwealth of the Northern Mariana Islands Public School System. 


Table D5. Hierarchical logistic regression model results for earning a score of Ready or Exceeding on the grade 
3 ACT Aspire reading assessment from analyses using multiple imputation of missing data and including 
interactions between ethnicity and Head Start enrollment 


Estimate 
NVe-Val-l 0) (=) (Ley=aere le} Sliclalollacm-lanelg 


Intercept —1.68*** 0.17 <.001 
Female 0.18** 0.06 <.01 
Received free or reduced-price lunch -0.21* 0.10 <.05 
Age at kindergarten entry (months) 0.04* 0.02 <.05 
Filipino 0.44*** 0.08 <.001 
Chamorro or Carolinian —0.33*** 0.08 <.001 
Had enrolled in Head Start -0.05 0.06 44 
Attended one school between kindergarten and grade 3 0.18* 0.08 <.05 
Filipino: Had enrolled in Head Start 0.04 0.08 59 
Chamorro or Carolinian: Had enrolled in Head Start 0.01 0.08 .86 


Cohort 0.27 na na 


* Significant at p < .05; ** significant at p < .01; *** significant at p < .001. 
na is not applicable. 
Note: n = 2,095. Variable names with : indicate an interaction term between two variables. 
Source: Authors’ analysis based on 2011-19 administrative data files provided by the Commonwealth of the Northern Mariana Islands Public School System. 
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Table D6. Hierarchical logistic regression model results for earning a score of Ready or Exceeding on the grade 
3 ACT Aspire reading assessment from analyses using multiple imputation of missing data and including 
interactions between ethnicity and Head Start enrollment and age at kindergarten entry and Head Start 
enrollment 


Estimate 
NVe-Val-) 0) (=1 (log odds) Standard error 


Intercept —1.68*** 0.17 <.001 
Female 0.18** 0.06 .001 
Received free or reduced-price lunch 0.21* 0.10 .03 
Age at kindergarten entry (months) 0.04* 0.02 01 
Filipino 0.44*** 0.08 <.001 
Chamorro or Carolinian —0.33*** 0.08 <.001 
Had enrolled in Head Start -0.05 0.06 45 
Attended one school between kindergarten and grade 3 0.18* 0.08 <.05 
Filipino: Had enrolled in Head Start 0.04 0.08 .60 
Chamorro or Carolinian: Had enrolled in Head Start 0.01 0.08 86 
Enrolled in Head Start: Age at kindergarten entry —0.004 0.02 79 
Rendometfet 
Cohort 0.29 na na 


* Significant at p < .05; ** significant at p < .01; *** significant at p < .001. 

na is not applicable. 

Note: n = 2,095. Variable names with: indicate an interaction term between two variables. 

Source: Authors’ analysis based on 2011-19 administrative data files provided by the Commonwealth of the Northern Mariana Islands Public School System. 


Supplementary analyses on Head Start enrollment 

The study team first calculated descriptive statistics for students who had enrolled in Head Start and for students 
who had not (table D7). Hierarchical logistic regression models were then used to estimate the associations 
between student demographic characteristics and the outcome of interest. For this analysis the outcome of 
interest was whether the student had enrolled in the CNMI Head Start program for one or more years. Predicted 
probabilities were then calculated from the results of the hierarchical models for demographic characteristics that 
had statistically significant associations with the outcome variable. The study team refers to associations as 
statistically significant if the p-value associated with the coefficient was less than .05. 


The main findings from these analyses show that: 


Filipino students were more likely to have enrolled in Head Start than students of other ethnicities. The 
predicted probability of having enrolled in Head Start was 67 percent for Filipino students compared with 45 
percent for Chamorro or Carolinian students and 46 percent for students who reported another ethnicity, 
after other student demographic characteristics were controlled for (tables D8 and D9). The difference in 
probability between Filipino students and both Chamorro or Carolinian students and students of other 
ethnicities was minor. 


Students who had enrolled in Head Start tended to enter kindergarten at an older age. Students who were 
older at kindergarten entry were more likely to have enrolled in Head Start than students who were younger 
at kindergarten entry, after other student demographic characteristics were controlled for (see tables D8 and 
D9). The predicted probability of having enrolled in Head Start was 63 percent for students who were 6 years 
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old (72 months) at kindergarten entry compared with 42 percent for students who were 4 years, 8 months 
old (56 months; see tables D8 and D9). The difference between those probabilities was major. 


e Student gender and free or reduced-price lunch receipt were not associated with Head Start enrollment. The 
odds of having enrolled in Head Start did not differ significantly between male and female students or 
between students who received free or reduced-price lunch and students who did not receive free or reduced- 
price lunch, after other student demographic characteristics and education experiences were controlled for 
(see table D8). 


Sample descriptives 


Table D7. Student demographic characteristics, by Head Start enrollment 


Had enrolled in 
Had notenrolledin one or more years 


All students Head Start of Head Start 
NVF-Val-]e)(=) (n = 2,659) (n=1,344) (n = 1,315) 


All students 100 50.6 49.5 


Male 50.7 51.2 50.3 
Female 48.9 48.1 49.6 
Gender missing 0.4 0.7 0.2 


Did not receive free or reduced-price lunch at kindergarten entry 89.5 90.7 88.3 


Received free or reduced-price lunch at kindergarten entry 10.5 9.3 11.7 


Mean age at kindergarten entry? (months) 64.4 (3.9) 64.1 (4.0) 64.7 (3.7) 


Filipino 32.0 22.0 42.4 
Chamorro or Carolinian 47.4 54.6 40.1 
Other 19.9 22.4 17.3 
Ethnicity missing 0.6 1.04 0.2 


Note: Percentages might not sum to 100 because of rounding. 

a. Information on age at kindergarten entry was missing for 90 students. Of those students who were missing information on age at kindergarten entry, 16.7 
percent had enrolled in one or more years of Head Start compared with 50.6 percent of those students for whom information on age was not missing. Values 
in parentheses are standard deviations, which apply only to continuous variables. 

Source: Authors’ analysis based on 2011-19 administrative data files provided by the Commonwealth of the Northern Mariana Islands Public School System. 


Head Start enrollment data analyses. This section provides detailed results of the regression analyses examining 
the relationship between Head Start enrollment and student demographic characteristics. 
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Table D8. Hierarchical logistic regression model results for predicting Head Start enrollment with multiple 
imputation for missing data 


Estimate 
NVF-Val-] 0)( =) (log odds) Standard error 


Intercept 0.12 0.16 43 
Female 0.04 0.04 36 
Received free or reduced-price lunch —0.13 0.07 .09 
Age at kindergarten entry (months) 0.05*** 0.01 <.001 
Filipino 0.58*** 0.06 <.001 
Chamorro or Carolinian —0.32*** 0.06 <.001 


School 0.41 na na 
Cohort 0.07 na na 


*** Significant at p < .001. 

na is not applicable. 

Note: n= 2,659. 

Source: Authors’ analysis based on 2011-19 administrative data files provided by the Commonwealth of the Northern Mariana Islands Public School System. 


Table D9. Predicted probabilities for Head Start enrollment with multiple imputation for missing data 


Subgroup Predicted probability 95 percent confidence interval 
Age at kindergarten entry—56 months 0.42 0.34—-0.50 
Age at kindergarten entry—58 months 0.44 0.37-0.52 
Age at kindergarten entry—60 months 0.47 0.40-0.55 
Age at kindergarten entry—62 months 0.50 0.42-0.57 
Age at kindergarten entry—64 months 0.53 0.45-0.60 
Age at kindergarten entry—66 months 0.55 0.48-0.63 
Age at kindergarten entry—68 months 0.58 0.50-0.65 
Age at kindergarten entry—70 months 0.61 0.53-0.68 
Age at kindergarten entry—72 months 0.63 0.55-0.71 
Filipino 0.67 0.59-0.74 
Chamorro or Carolinian 0.45 0.38-0.53 
Other ethnicities 0.46 0.38-0.55 


Note: n = 2,659. Only variables identified as statistically significant in table D8 are included in this table. 
Source: Authors’ analysis based on 2011-19 administrative data files provided by the Commonwealth of the Northern Mariana Islands Public School System. 


Head Start enrollment sensitivity analyses. The results of the analysis using complete case analysis and the analysis 
using imputed data and the results of the analysis with all values for age at kindergarten entry and the analysis in 
which extreme age values were removed were similar for supplementary analyses on Head Start enrollment, with 
differences on all predictors less than or equal to 0.25 log odds. These sensitivity analyses found that Filipino 
students and students who were older for their grade had a statistically significantly higher probability of having 
enrolled in Head Start than the average student. Both multiple imputation and complete case analyses also found 
that age at kindergarten entry is positively and statistically significantly associated with Head Start enrollment 
(tables D10-D13). 
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Table D10. Hierarchical logistic regression model results for Head Start enrollment using multiple imputation 
for missing data 


Multiple imputation Complete cases 
(n = 2,659) (n = 2,594) 


Simlarerlae| Slrlaler-] ae] 
Variable Estimate lanel Estimate error 


Intercept 0.12 0.16 43 0.14 0.15 39 
Female 0.04 0.04 36 0.03 0.04 38 
Received free or reduced-price lunch -0.13 0.07 .09 —0.12 0.08 11 
Age at kindergarten entry (months) 0.05*** 0.01 <.001 0.06*** 0.01 <.001 
Filipino 0.58*** 0.06 <.001 0.56*** 0.06 <.001 
Chamorro or Carolinian —0.32*** 0.06 <.001 —0.29*** 0.06 <.001 


School 0.41 na na 0.43 na na 
Cohort 0.07 na na 0.03 na na 


*** Significant at p < .001. 
na is not applicable. 
Source: Authors’ analysis based on 2011-19 administrative data files provided by the Commonwealth of the Northern Mariana Islands Public School System. 


Table D11. Predicted probability of Head Start enrollment using multiple imputation for missing data and 
complete case analysis 


Multiple imputation Complete case 
(n = 2,659) (n = 2,594) 


Predicted 95 percent Predicted 95 percent 


NYU] off gel 0] 0) probability confidence interval probability confidence interval 


Age at kindergarten entry—56 months 0.42 0.34—0.50 0.42 0.34-0.51 
Age at kindergarten entry—58 months 0.44 0.37-0.52 0.45 0.37-0.53 
Age at kindergarten entry—60 months 0.47 0.40-0.55 0.48 0.40-0.55 
Age at kindergarten entry—62 months 0.50 0.42-0.57 0.50 0.43-0.58 
Age at kindergarten entry—64 months 0.53 0.45-0.60 0.53 0.46-0.60 
Age at kindergarten entry—66 months 0.55 0.48-0.63 0.56 0.48-0.63 
Age at kindergarten entry—68 months 0.58 0.50-0.65 0.59 0.51-0.66 
Age at kindergarten entry—70 months 0.61 0.53-0.68 0.61 0.53-0.69 
Age at kindergarten entry—72 months 0.63 0.55-0.71 0.64 0.56—-0.71 
Filipino 0.67 0.59-0.74 0.67 0.59-0.74 
Chamorro or Carolinian 0.45 0.38-0.53 0.46 0.59-0.74 
Other ethnicities 0.46 0.38-0.55 0.47 0.39-0.55 


Source: Authors’ analysis based on 2011-19 administrative data files provided by the Commonwealth of the Northern Mariana Islands Public School System. 


Table D12. Hierarchical logistic regression model results for Head Start enrollment for the full sample and 
excluding students with values for age at kindergarten entry in the <2.5 and >97.5 percentiles 


All values for Excluding students with values for 
age at kindergarten entry age at kindergarten entry 
Tare lerel=xe} in the <2.5 and >97.5 percentiles 
(n = 2,659) (n = 2,618) 
Standard S\mlalet-lae} 
Variable Estimate error Estimate lagelg 
Fixed effect 
Intercept 0.12 0.16 .43 0.13 0.15 39 
Female 0.04 0.04 36 0.04 0.04 34 
Received free or reduced-price lunch -0.13 0.07 .09 0.12 0.07 10 
Age at kindergarten entry (months) 0.05*** 0.01 <.001 0.05*** 0.01 <.001 
Filipino 0.58*** 0.06 <.001 0.58*** 0.06 <.001 
Chamorro or Carolinian —0.32*** 0.06 <.001 —0.32*** 0.06 <.001 


Random effect 
School 0.41 na na 0.41 na na 
Cohort 0.07 na na 0.07 na na 


*** Significant at p < .001. 
na is not applicable. 
Source: Authors’ analysis based on 2011-19 administrative data files provided by the Commonwealth of the Northern Mariana Islands Public School System. 


Table D13. Predicted probability of Head Start enrollment for the full sample and excluding students with 
values for age at kindergarten entry in the <2.5 and >97.5 percentiles 


All values for Excluding students with values for 
age at kindergarten entry age at kindergarten entry 
Tare lerel=xe} in the <2.5 and >97.5 percentiles 
(n = 2,659) (n = 2,618) 


Predicted 95 percent acvel(ancye| 95 percent 
NYU] oi gel 0] 0) probability confidence interval probability confidence interval 


Student characteristics 


Age at kindergarten entry—56 months 0.42 0.34-0.50 0.42 0.34-0.50 
Age at kindergarten entry—58 months 0.44 0.37-0.52 0.45 0.37-0.53 
Age at kindergarten entry—60 months 0.47 0.40-0.55 0.47 0.40-0.55 
Age at kindergarten entry—62 months 0.50 0.42-0.57 0.50 0.43-0.57 
Age at kindergarten entry—64 months 0.53 0.45—0.60 0.53 0.46-0.60 
Age at kindergarten entry—66 months 0.55 0.48-0.63 0.56 0.48-0.63 
Age at kindergarten entry—68 months 0.58 0.50-0.65 0.58 0.51-0.65 
Age at kindergarten entry—70 months 0.61 0.53-0.68 0.61 0.53-0.68 
Age at kindergarten entry—72 months 0.63 0.55-0.71 0.63 0.55-0.71 
Filipino 0.67 0.59-0.74 0.67 0.60—0.74 
Chamorro or Carolinian 0.45 0.38-0.53 0.45 0.38-0.53 
Other ethnicities 0.46 0.38-0.55 0.47 0.39-0.55 


Source: Authors’ analysis based on 2011-19 administrative data files provided by the Commonwealth of the Northern Mariana Islands Public School System. 


